3D tracking of laparoscopic instruments using statistical and geometric modeling

International audienceDuring a laparoscopic surgery, the endoscope can be manipulated by an assistant or a robot. Several teams have worked on the tracking of surgical instruments, based on methods ranging from the development of specific devices to image processing methods. We propose to exploit the instruments' insertion points, which are fixed on the patients abdominal cavity, as a geometric constraint for the localization of the instruments. A simple geometric model of a laparoscopic instrument is described, as well as a parametrization that exploits a spherical geometric grid, which offers attracting homogeneity and isotropy properties. The general architecture of our proposed approach is based on the probabilistic Condensation algorithm

Multi-View Vision System for Laparoscopy Surgery.

International audienceThis paper deals with the development of a new generation of augmented laparoscopy system. We propose to equip a traditional endoscope, or a robotic endoscope holder, with a miniature stereovision device. The system includes two miniature high resolution CMOS cameras mounted around the endoscope as a pair of glasses that provides a global view of the abdominal cavity completing the traditional view. Each camera can reach a frame rate of 30 images/second with a resolution of 1600 _ 1200 pixels. A deployment, fixation and rapid extraction system of the proposed device through the trocar was designed and validated through preclinical experiments (testbench and human cadaver). The main benefit of the proposed system in the minimally invasive surgery domain is to provide simultaneously local/global views, and with perspectives in 3D reconstruction of the organ being treated

A model for predicting dissolved organic carbon distribution in a reservoir water using fluorescence spectroscopy

A number of water treatment works (WTW) in the north of England (UK) have experienced problems in reducing the dissolved organic carbon (DOC) present in the water to a sufficiently low level. The problems are experienced in autumn/ winter when the colour increases and the coagulant dose at the WTW needs to be increased in order to achieve sufficient colour removal. However, the DOC content of the water varies little throughout the year. To investigate this further, the water was fractionated using resin adsorption techniques into its hydrophobic (fulvic and humic acid fractions) and hydrophilic (acid and non-acid fractions) components. The fractionation process yields useful information on the changing concentration of each fraction but is time consuming and labour intensive. Here, a method of rapidly determining fraction concentration was developed using fluorescence spectroscopy. The model created used synchronous spectra of fractionated material compared against bulk water spectra and predicted the fraction concentrations to within 10% for a specific water. The model was unable to predict fraction concentrations for waters from a different watershed

Visual servoing of a robotic endoscope holder based on surgical instrument tracking

International audienceWe propose an image-based control for a roboticendoscope holder during laparoscopic surgery. Our aim is toprovide more comfort to the practitioner during surgery byautomatically positioning the endoscope at his request. To doso, we propose to maintain one or more instruments roughly atthe center of the laparoscopic image through different commandmodes. The originality of this method relies on the direct useof the endoscopic image and the absence of artificial markersadded to the instruments. The application is validated on a testbench with a commercial robotic endoscope holder

Using CamiTK for rapid prototyping of interactive Computer Assisted Medical Intervention applications

Computer Assisted Medical Intervention (CAMI hereafter) is a complex multi-disciplinary field. CAMI research requires the collaboration of experts in several fields as diverse as medicine, computer science, mathematics, instrumentation, signal processing, mechanics, modeling, automatics, optics, etc

Development of miniaturized light endoscope-holder robot for laparoscopic surgery

PURPOSE: We have conducted experiments with an innovatively designed robot endoscope holder for laparoscopic surgery that is small and low cost. MATERIALS AND METHODS: A compact light endoscope robot (LER) that is placed on the patient's skin and can be used with the patient in the lateral or dorsal supine position was tested on cadavers and laboratory pigs in order to allow successive modifications. The current control system is based on voice recognition. The range of vision is 360 degrees with an angle of 160 degrees . Twenty-three procedures were performed. RESULTS: The tests made it possible to advance the prototype on a variety of aspects, including reliability, steadiness, ergonomics, and dimensions. The ease of installation of the robot, which takes only 5 minutes, and the easy handling made it possible for 21 of the 23 procedures to be performed without an assistant. CONCLUSION: The LER is a camera holder guided by the surgeon's voice that can eliminate the need for an assistant during laparoscopic surgery. The ease of installation and manufacture should make it an effective and inexpensive system for use on patients in the lateral and dorsal supine positions. Randomized clinical trials will soon validate a new version of this robot prior to marketing

Medical image computing and computer-aided medical interventions applied to soft tissues. Work in progress in urology

Until recently, Computer-Aided Medical Interventions (CAMI) and Medical Robotics have focused on rigid and non deformable anatomical structures. Nowadays, special attention is paid to soft tissues, raising complex issues due to their mobility and deformation. Mini-invasive digestive surgery was probably one of the first fields where soft tissues were handled through the development of simulators, tracking of anatomical structures and specific assistance robots. However, other clinical domains, for instance urology, are concerned. Indeed, laparoscopic surgery, new tumour destruction techniques (e.g. HIFU, radiofrequency, or cryoablation), increasingly early detection of cancer, and use of interventional and diagnostic imaging modalities, recently opened new challenges to the urologist and scientists involved in CAMI. This resulted in the last five years in a very significant increase of research and developments of computer-aided urology systems. In this paper, we propose a description of the main problems related to computer-aided diagnostic and therapy of soft tissues and give a survey of the different types of assistance offered to the urologist: robotization, image fusion, surgical navigation. Both research projects and operational industrial systems are discussed

Comparative evaluation of instrument segmentation and tracking methods in minimally invasive surgery

Intraoperative segmentation and tracking of minimally invasive instruments is a prerequisite for computer- and robotic-assisted surgery. Since additional hardware like tracking systems or the robot encoders are cumbersome and lack accuracy, surgical vision is evolving as promising techniques to segment and track the instruments using only the endoscopic images. However, what is missing so far are common image data sets for consistent evaluation and benchmarking of algorithms against each other. The paper presents a comparative validation study of different vision-based methods for instrument segmentation and tracking in the context of robotic as well as conventional laparoscopic surgery. The contribution of the paper is twofold: we introduce a comprehensive validation data set that was provided to the study participants and present the results of the comparative validation study. Based on the results of the validation study, we arrive at the conclusion that modern deep learning approaches outperform other methods in instrument segmentation tasks, but the results are still not perfect. Furthermore, we show that merging results from different methods actually significantly increases accuracy in comparison to the best stand-alone method. On the other hand, the results of the instrument tracking task show that this is still an open challenge, especially during challenging scenarios in conventional laparoscopic surgery

Vers la commande en boucle fermée d'un robot porte-optique : apport de l'analyse contextuelle d'images endoscopiques.

The development of laparoscopy is closely linked to the advances in instrumentation, allowing to combine the efficacy of surgery and a minimally invasive approach. Today, robotic systems can assist a surgeon during a minimally invasive intervention, but their use is still controversial because of their bulkiness, their cost, and the change they might induce to surgical practice. Research trends in this field aim towards a better adaptation of these systems to surgical constraints.The objective of this Ph.D. Thesis was to control a robotized endoscopic camera holder thanks to " high-level " commands mimicking the task of the assistant in charge of manipulating the endoscope in conventional laparoscopy. First, I built a model of the field of laparoscopic surgery highlighting the role of the assistant. It allowed me to identify the commands primitives to develop. Then, I worked on the conception and development of the selected command primitives, especially a command dedicated to the robot's automatic tracking of surgical instruments based on the analysis of the endoscopic images. The instrument detection algorithm was first validated on numerized images extracted from surgical procedures. Then a cadaver experiment was performed, during which we were able to track a surgical instrument with the robot, in conditions close to clinical conditions.Les progrès de l'instrumentation ont été indispensables au développement de la cœlioscopie, permettant de combiner l'efficacité de la chirurgie avec une voie d'abord mini-invasive. Les systèmes robotisés font maintenant partie de l'arsenal dans l'assistance à la réalisation d'une intervention par technique mini-invasive, mais leur intérêt reste encore controversé en raison de leur encombrement, leur coût et les transformations qu'ils risquent d'imposer à la pratique chirurgicale. Les problématiques de recherche dans le domaine de la cœlioscopie robotisée visent une meilleure adaptation de ces systèmes aux contraintes chirurgicales.L'objectif de cette thèse était de contrôler un robot porte-endoscope grâce à primitives de commandes " haut-niveau " qui imitent la tâche de l'assistant chargé de manipuler l'endoscope en cœlioscopie classique. Dans un premier temps, j'ai construit un modèle du domaine de la chirurgie cœlioscopie mettant en évidence le rôle de l'assistant qui m'a permis de déterminer les commandes pertinentes pour contrôler un robot porte-endoscope. Je me suis ensuite intéressée à la mise en œuvre de ces commandes, notamment l'une d'entre elles, consacrée au suivi automatique, par le robot, des instruments chirurgicaux à partir de l'analyse des images endoscopiques. La détection des instruments chirurgicaux a d'abord été validée sur des images numérisées de procédures chirurgicales, puis nous avons réalisé un test sur cadavre, plus proche de la réalité clinique, au cours duquel nous avons réussi à suivre un instrument chirurgical avec le robot porte endoscope

Vers la commande en boucle fermée d'un robot porte-optique : apport de l'analyse contextuelle d'images endoscopiques.

The development of laparoscopy is closely linked to the advances in instrumentation, allowing to combine the efficacy of surgery and a minimally invasive approach. Today, robotic systems can assist a surgeon during a minimally invasive intervention, but their use is still controversial because of their bulkiness, their cost, and the change they might induce to surgical practice. Research trends in this field aim towards a better adaptation of these systems to surgical constraints.The objective of this Ph.D. Thesis was to control a robotized endoscopic camera holder thanks to " high-level " commands mimicking the task of the assistant in charge of manipulating the endoscope in conventional laparoscopy. First, I built a model of the field of laparoscopic surgery highlighting the role of the assistant. It allowed me to identify the commands primitives to develop. Then, I worked on the conception and development of the selected command primitives, especially a command dedicated to the robot's automatic tracking of surgical instruments based on the analysis of the endoscopic images. The instrument detection algorithm was first validated on numerized images extracted from surgical procedures. Then a cadaver experiment was performed, during which we were able to track a surgical instrument with the robot, in conditions close to clinical conditions.Les progrès de l'instrumentation ont été indispensables au développement de la cœlioscopie, permettant de combiner l'efficacité de la chirurgie avec une voie d'abord mini-invasive. Les systèmes robotisés font maintenant partie de l'arsenal dans l'assistance à la réalisation d'une intervention par technique mini-invasive, mais leur intérêt reste encore controversé en raison de leur encombrement, leur coût et les transformations qu'ils risquent d'imposer à la pratique chirurgicale. Les problématiques de recherche dans le domaine de la cœlioscopie robotisée visent une meilleure adaptation de ces systèmes aux contraintes chirurgicales.L'objectif de cette thèse était de contrôler un robot porte-endoscope grâce à primitives de commandes " haut-niveau " qui imitent la tâche de l'assistant chargé de manipuler l'endoscope en cœlioscopie classique. Dans un premier temps, j'ai construit un modèle du domaine de la chirurgie cœlioscopie mettant en évidence le rôle de l'assistant qui m'a permis de déterminer les commandes pertinentes pour contrôler un robot porte-endoscope. Je me suis ensuite intéressée à la mise en œuvre de ces commandes, notamment l'une d'entre elles, consacrée au suivi automatique, par le robot, des instruments chirurgicaux à partir de l'analyse des images endoscopiques. La détection des instruments chirurgicaux a d'abord été validée sur des images numérisées de procédures chirurgicales, puis nous avons réalisé un test sur cadavre, plus proche de la réalité clinique, au cours duquel nous avons réussi à suivre un instrument chirurgical avec le robot porte endoscope